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Using clickers in large college psychology classes: Academic 
achievement and perceptions 

Selma Powell 1 , Carrie Straub, Jacqueline Rodriguez, and Barbara VanHorn 2 

Abstract: This research study explored the effects of the use of clicker technology 
as a means of formative assessment in large, college introductory psychology 
courses. Academic achievement, as measured by performance on tests of 
knowledge, was compared between students who used clickers and students that 
did not use clickers. There was a significant difference between the two groups, 
although a small effect was noted. Students using the clickers responded to survey 
items created to solicit information regarding student perceptions of increased 
understanding, ability to self-identify content deficits, fun, and financial value of 
clickers. Data from the survey indicated that the majority of students perceived 
the usage of clickers increased their understanding of and ability to self-identify 
concept areas. All of the students reported that using clickers was fun, although 
approximately only one third of those same students reported that purchasing a 
clicker was worthwhile. Implications for administrators are discussed and 
alternative technologies are explored. 
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It is not uncommon for undergraduates in public colleges and universities to find themselves 
faced with their first college lecture in an auditorium-style classroom (Kenwright, 2009; Mayer 
et ah, 2009). Lecture-hall classrooms accommodate large numbers of learners, and lessons are 
delivered using a teacher-centered lecture format. Within this context, researchers and faculty 
from a small public college in the southeast United States located in a diverse urban community 
sought to establish effective instructional supports for large, lecture-style classes, specifically 
through the use of class wide electronic response systems. The purpose of this research is to 
examine the use of clicker technology on student achievement and perception. In the spirit of 
Boyer’s (1990) call to action, the research team examined classroom teaching practice as a 
means of “transforming and extending” the knowledge base for instructors with large class 
enrollments (p.24). 

I. Background. 

Instructors of mass classes rely on traditional, lecture-style lessons to deliver information to large 
numbers of students; however this lesson format provides little or no interaction or discussion 
(Geske, 1992; Gleason, 1986). Although large lecture halls may accommodate more students by 
making efficient use of faculty and facility resources, student understanding of content can be 
compromised. This learning environment may seem passive and impersonal for undergraduates 
(Hoekstra, 2008). Hall, Collier, Thomas, and Hilgers (2005) identified two problems that 
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students in mass class settings face: (1) lack of engagement in the lesson and (2) lack of meta- 
cognitive awareness or ability to self-identify deficits in subject area awareness. The lack of 
engagement coupled with the students’ inability to self-identify deficits in subject area 
knowledge may result in an inability to master course content. Furthermore, Edmonds and 
Edmonds (2008) stated that mass class learning environments do not provide opportunity for 
instructors to gauge students’ understanding of subject areas until the exam results are presented, 
a scenario leaving instructors and students in the dark regarding subject area understanding until 
it is too late to remediate misconceptions. 

According to generative theory of learning, learners actively engage in cognitive 
processing during learning by attending to relevant material, mentally organizing the selected 
material, and integrating the organized material with prior knowledge (Mayer, 2001; Mayer et 
ah, 2009; Wittrock, 1974). Therefore, instructors optimize learning by engaging in instructional 
activities which “prime active cognitive processing in learners” (Mayer et ah, 2009, p.53). The 
research on generative learning theory indicates that the outcome of active cognitive processing 
is a meaningful learning outcome (Mayer et al., 2009). Anderson et al. (2001) theorize that 
learning outcomes, including the knowledge and skills learned in class, can be evaluated using a 
variety of measures, including academic test items of class content. 

To foster the generative learning process, instructors may use questioning methods (King, 
1992; Rosenshine, Meister, & Chapman, 1996). Yet, faced with the challenge of assessing 
student knowledge and increasing engagement in mass classrooms, how do instructors gain 
knowledge about student understanding to provide responsive instruction? In classes with 
smaller groups of students, instructors may use questioning methods to assess student 
understanding, as incorrect student responses provide opportunity for instructors to provide 
corrective feedback to change student misconceptions (Kenwright, 2009; Mayer et al., 2009). 
However, in large, lecture-style class’s students may be intimidated by asking or responding to 
questions (Poirier & Feldman, 2007). One method for increasing student understanding and 
engagement is through the use of electronic clickers, a tool for generative learning. 

Clickers, also referred to as electronic response systems, personal response systems, 
audience response systems, classroom communication systems, or student response systems, 
allow students to individually respond to closed-ended questions posed by the instructor. 
Anonymous student responses are gathered via a remote handheld device, commonly referred to 
as a “clicker”, and responses are immediately provided to the instructor. The instructor may then 
broadcast the results to the class using a multi-media projection device. Publicly examining 
student responses achieves a variety of objectives, which include but are not limited to, 
assessment of student understanding, increasing student engagement in the lesson, and providing 
students with a venue for objectively self-assessing their knowledge of the subject area (Mayer et 
al., 2009). The clicker system is comprised of two parts: a classroom-wide response system and a 
set of individual, handheld, remote clicker devices. Colleges generally incur several thousand 
dollars in initial cost to install the classroom-wide response system, passing along the cost of the 
reusable handheld clicker device (between $10 and $40) to students. Periodic maintenance as 
well as technical support costs are incurred by the college over time. 

Clicker-based pedagogy has the potential to counteract the passive, teacher-centered 
instruction often encountered in mass classes of today’s colleges and universities. Teachers and 
students may benefit from the real-time assessment of subject area knowledge. Additionally, 
increased student engagement in class may result in increased perceptions of fun, providing 
motivation to return to class. Although there is evidence to suggest that clickers increase 
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academic achievement, as measured by performance on tests of knowledge, as well as provide 
ability to assess student knowledge, and increase student perceptions of fun, it is also critical to 
understand student perceptions of the value of clickers as a learning tool. 

A. Utilization Of Clicker-Based Pedagogy Increases Academic Achievement. 

A growing body of research has been used to provide evidence for increased academic 
achievement, as measured by perfonnance on tests of knowledge, using clickers. Active learning 
has been linked to increased student achievement, which can be measured by final grades 
(United States Department of Education, 1996). A study of 11 parallel courses taught at the 
University of Wisconsin over two years showed a statistically significant impact of clicker use 
on student academic achievement (Kaleta & Joosten, 2007). Similarly, Salmonson, Andrew, and 
Everett (2009) reported that increasing the engagement of non-participatory students through the 
use of clickers resulted in an increase in grades. El-Rady (2006) found statistically significant 
differences in exam scores between classes compared in two consecutive semesters and 
furthermore found evidence to suggest that clickers improved student retention. In a comparison 
of clicker technology versus paper pencil technology, Mayer et al. (2009) found that students 
who experienced in-class questioning and responded with paper and pencil scored significantly 
lower on class exams than those who responded using clickers. Therefore, the research team 
hypothesized that the use of clickers would increase students’ performance. Thus, the first 
research question is: 

RQi: To what extent does the use of clickers increase student academic achievement in a 
large, college-level introductory psychology class as measured by final grade percentage score? 

B. Increased Understanding Of Content Through Formative Assessment. 

Clickers ostensibly serve as vehicles to increase student understanding of content by providing 
instructors with the opportunity to take on the role of agile teachers who can quickly assess 
student comprehension and modify instruction to student needs (Beatty, Garace, Leonard & 
Dufrense, 2006; Mayer et al, 2009; Mula & Kavanaugh, 2009). Researchers distinguish between 
pedagogy and technology, highlighting the clicker’s role as a piece of formative assessment 
technology (Beatty et al., 2006), which provides “college instructors with a non-intrusive, 
effective pedagogy and students with a more engaging learning format” (Mula & Kavanagh, 
2009, p. 2). Researchers suggest that clickers alone do not increase academic achievement, but 
rather the utilization of clickers increases opportunities for instructors to engage students by 
using interactive questioning methods which, in turn, are related to increased academic 
achievement (Mayer et al., 2009). Kaleta and Joosten (2007) reported survey results yielding that 
100% of instructors appreciated the ability to assess student knowledge and understanding, while 
74% agreed or strongly agreed that clickers improved student learning. The research team 
hypothesized that students using clicker technology would perceive an increase in understanding 
of the topic based on the use of clickers. Thus, the second research question is: 

RQ 2 : Did students using clicker technology in a large, college-level introductory 
psychology class perceive that clickers increased “understanding of the topic”? 
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C. Using Clickers To Self-Identify Content Knowledge Deficits. 

While instructors report value in assessing understanding of content, clicker responses also 
provide valuable information for students to identify personal deficits in understanding. In an 
examination of meta-cognition skills of individuals, Flavell (1979) identified limitations in the 
ability to self-monitor comprehension, reporting that individuals were often unable to identity 
holes in understanding. The use of clickers may reveal misconceptions to students; when 
instructors broadcast anonymous responses to questions using clickers, students are able to 
compare individual responses to the correct answers in an unobtrusive setting. In this way, each 
student has the opportunity to practice formative assessment at the individual level (Beatty et al., 
2006). Student-identified deficits in content knowledge have the potential to allow students to 
create a customized, focused plan of study. Clickers may provide one method for students to 
gather objective information about their learning. The research team hypothesized that students 
using clicker technology would perceive an increase in their ability to identify areas of deficit in 
their subject matter knowledge. Thus, the third research question is: 

RQ3: Did students using clicker technology in a large, college-level introductory 
psychology class perceive that the use of clickers helped them find out what they “still needed to 
study”? 

D. Using Clickers to Increase Student Perception of Fun. 

Although there is minimal research related to student perception of fun in a clicker-based 
classroom (Fies & Marshall, 2006), investigating student perceptions of clicker use is worthwhile 
to establish social validity of the utilization of clickers (Wolf, 1978). Beatty et al. (2006) writes, 
“By fostering an active, interactive classroom environment, classroom communication system- 
based pedagogy helps keeps students interested and attentive” (p. 6). In a large introductory 
psychology class, Poirier and Feldman (2007) found that students using clickers earned higher 
final exam scores and reported positive attitudes toward utilizing clickers in class. Researchers 
report that students find clicker-based classes to be more fun than non-clicker based classes 
(Bumstein & Ledennan, 2001; Dufresne, Gerace, Leonard, Mestre, & Wenk, 1996; Fies, 2005). 
Based on these results, the research team hypothesized that students using clicker technology 
would perceive using clickers as fun. Thus, the fourth research question is: 

RQ 4 : Did students using clicker technology in a large, college-level introductory 
psychology class perceive that the use of clickers to respond to questions was “fun”? 

E. Financial Value of Clickers. 

A primary concern of colleges and universities using the clickers is the issue of cost. Students 
enrolled in classes using clicker-based pedagogy are required to purchase the handheld clicker 
device that may cost between $10 and $40. Students may re-use the device each semester, but if 
lost, students must purchase and register another. Although some researchers report that students 
“appreciate the system’s value” (Beatty, 2004, p.6), researchers of two recent literature reviews 
of clicker technology did not discuss whether or not students perceived the clicker was worth 
purchasing (Fies & Marshall, 2006; Roschelle, Abrahamson, & Penuel, 2004). Because students 
must incur the cost of utilizing clickers after the decision to invest in the system has already been 
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made by administrators, it is important to investigate whether or not students perceive the clicker 
is worth purchasing to establish social validity of the utilization of clickers (Wolf, 1978). The 
research team hypothesized that students using clicker technology would indicate that they 
would be willing to purchase a clicker for a class to help them leam. Thus, the fifth research 
question is: 

RQs: Did students using clicker technology indicate that they would be “willing to 
purchase a clicker that cost $30 - $40” to help them learn? 

II. Methodology. 

A. Participants. 

Participants in this study were enrolled in an urban public college located in the southeast United 
States in introductory psychology classes and were grouped according the section of the course 
in which they were enrolled. The same instructor taught both of the sections sampled for this 
study. The sample (N = 145) was established when students met the criteria of enrollment in the 
introductory course in psychology and completion of the final exam. Students self-selected their 
participation in the study and were not provided monetary compensation, nor were they promised 
any benefits not normally associated with using a learning tool. Participant information was 
numerically coded to maintain anonymity and all identifiable information was maintained in a 
secure location by the principal investigator. Participation for this study totaled 183 students 
across two separate introductory psychology classes and students were assigned into control and 
treatment groups. Of the 183 participating students, 145 students received a final grade in the 
course and were subsequently involved in the analysis. The control group did not receive clickers 
and consisted of 78 students from one class. The treatment group received clickers at varied 
intervals and consisted of 67 participants from two other classes. Participants responded to a 
demographic questionnaire, infonnation on gender was broken down as follows: 56 of the 
participants were male while 127 of the participants were female, while responses to questions 
about ethnicity were as follows: Caucasian, 91; African-American, 52; Hispanic, 24; Asian, 2; 
Other, 14. Only 7 of the participants were under the age of 18, while the preponderance of the 
participants were between the ages of 18-24 (152), and 24 students were over 25 years old. 

B. Instruments. 

Primary outcomes were academic and measured by final class grades. For the purposes of this 
study, a common grading rubric was used for both of the introductory classes as defined in 
PASW as: A = 100-90%, B = 89-80%, C = 79-70%, D = 69-60%, and F = 59% or less. 

Data on student perception was collected using a researcher-created questionnaire 
comprised of 11 items with responses on a 5-point Likert scale. Questions selected for use in this 
research study were as follows: a) Using a clicker increased my understanding of the topic, b) 
Using a clicker helped me find out what I still needed to study, c) Using a clicker to respond to 
questions was fun, and d) I would be willing to purchase a clicker that cost $30-$40 to help me 
leam. Likert responses were as follows: 1) I strongly disagree, 2) I somewhat disagree, 3) 
Neutral, 4) I somewhat agree, and 5) I strongly agree. Demographic information for all 
participants was collected using a 21-question survey at the beginning of the course. At the 


Journal of the Scholarship of Teaching and Learning, Vol. 11, No. 4, December 2011. 
www.iupui.edu/~josotl 


5 



Powell, S., Straub, C., Rodriguez, J., and VanHorn, B. 


completion of the course, the group who used the clicker technology completed a questionnaire 
that consisted of 10 items related to attitude toward using the clickers. 

C. Data Collection and Analysis Procedures. 

In this quasi-experimental study, measures of academic perfonnance were compared across two 
groups of students: one class utilizing clickers (consisting of embedded questions and procedures 
for responding using clickers) and one class that did not utilize clickers. The same doctoral level 
professor with 10 years of experience taught each class, and each of the classes participating in 
this study met over the course of a semester. In all the courses, content was delivered via 
PowerPoint and lecture in a face-to-face setting. For the course with embedded questions, the 
average percentage of question to content slides was 21%, and the professor utilized clickers 
throughout the delivery of the lecture. Upon response to question, results were immediately 
displayed on a bar graph. If the distribution showed 10% or more of incorrect student responses, 
the feedback would be used to re-teach the topic by providing additional explanations or 
responding to questions. If more than 90% of students chose the correct response, the instructor 
would confirm the correct response, giving a brief explanation and then moving on. In both 
courses, online review quizzes allowed students to review content prior to administering class 
exams. The clickers system employed for the study was the iClicker manufactured in partnership 
with the Worth-Freeman Publishing Company. 

III. Findings. 

Participants in all the psychology classes totaled 183 students. Of the total student participation, 
145 received a final grade for the course, 78 of the participants were in the control group and 67 
participants were in the experimental group. Due to missing data, 38 cases were not included in 
the analysis. Thus, data was analyzed using 78.2% of the 183 total students (N=145). Using 
Predictive Analytics Software (PASW), the researchers compared the 38 missing cases to 
detennine if the ratios of missing data were similar between groups. Results yielded similar rates 
of missing data when compared across groups. Of the 38 cases, 20 cases of missing data 
occurred in the control group (accounting for 20.5% of the total control group), while 18 cases of 
missing data occurred in the experimental group (accounting for a loss of 21.2% of the 
experimental group). Inferential statistical analysis was perfonned for the first research question 
comparing experimental and control groups. Descriptive statistics on the perception of clicker 
technology usage were collected on the experimental group only for research questions two 
through five. 

RQi: To what extent does the use of clickers increase student performance in a large, 
college-level introductory psychology class as measured by final grade percentage score? 

Student performance was measured by final grades, defined by a percentage score of 
cumulative total points earned divided by cumulative total points possible. An independent t test 
was conducted to determine if there was a significant difference {p < 0.05) in mean score of final 
percentage course grades of students in a college-level introductory psychology class using a 
clicker system versus students who did not using a clicker system. The test was conducted using 
an alpha of 0.05. Levene’s test indicated that the assumption of homogeneity of variances was 
met (.F = 2.029, p = 0.157). The test was statistically significant, t(143) = 8.367, p < 0.05. 
Students using the clicker system scored higher on average (M = 83.43, SD = 11.88) than 
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students not using the clicker system (M= 76.82, SD = 15.13). The 95% confidence interval for 
the difference between means was -11.13 to -2.09. The effect size was calculated by eta squared 
and found to be 0.055 indicating that approximately 6% of the variance in scores was accounted 
for by whether or not the student used clickers. Approximately 81.27% of the treatment group (n 
= 67) received a grade of C or better compared to 76.92% of the non-treatment group (n = 78). 
The results provide evidence to support the hypothesis that students using clickers score higher 
in measures of academic performance. 

RQ 2 : Did students using clicker technology in a large, college-level introductory 
psychology class perceive that clickers increased “understanding of the topic”? 

In order to address the second research question, the questionnaire results (n = 67) from 
the treatment group were dichotomized based on responses. The responses of 1 or 2 were 
classified as “disagree” and the responses of 4 or 5 were classified as “agree,” and the response 
of 3 was neutral and not included in this comparison. Based on analysis, approximately 91% of 
students perceive that clickers increased their understanding of a topic, leaving only 9% of 
students in disagreement with the statement that using clickers increased their understanding. 
The survey results provide evidence to support the hypothesis that the majority of students 
perceive that using clickers increase their understanding of the topic. 

RQ3: Did students using clicker technology in a large, college-level introductory 
psychology class perceive that the use of clickers helped them find out what they “still needed to 
study”? 

In response to the third research question and based on the dichotomized responses, 
approximately 81% of the students (n = 67) who responded to the survey agreed that clickers 
improved their ability to identify their individual deficits in the subject matter during the lecture. 
Only approximately 6% disagree that clickers had an impact on their ability to identify their 
individual deficits in the content. Once again, the survey results support the hypothesis that the 
majority of students perceive that clickers improved their ability to self-identify subject matter 
they still need to study. 

RQr Did students using clicker technology in a large, college-level introductory 
psychology class perceive that the use of clickers to respond to questions was “fun”? 

Based on the dichotomized results of the survey, the fourth research question had an 
overwhelming positive response, with 100% of the students (n = 67) reporting that the use of 
clickers was fun as related to learning the content in an introductory Psychology course. The 
results of the survey support the hypothesis that using clickers in the course was fun. 

RQs: Did students using clicker technology indicate that they would be “willing to 
purchase a clicker that cost $30 to $40” to help them leam? 

Finally, for the last research question, the responses were dichotomized, however, this 
time the majority, approximately 35% of those who responded to the survey (n = 67) disagree 
that purchasing a clicker was worthwhile, with approximately 32% of students agreeing that 
purchasing a clicker was worthwhile. The results of this question on this survey were not in line 
with the original hypothesis. 

IV. Conclusion. 

Overall, there is evidence to suggest that the use of clickers in a large, college level introductory 
psychology class contributed positively to learning. Students using clickers performed 
significantly better when compared with a group of students not using clickers, although the 
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effect size was minimal. Students using clicker technology reported that they felt utilization of 
clickers increased their understanding of the topic and allowed them to self-identify areas of 
deficit in their learning. All of the students in the experimental group agreed that using clickers 
to respond to questions was fun; however, only approximately one-third of those same students 
thought that purchasing a clicker was worthwhile. While these students may have benefited from 
and enjoyed using the clickers, they do not seem to be willing to bear the expense. 

This study contained limitations that may bring in to question the validity of the findings. 
The results may not be generalizable to other instructional settings because this research was 
conducted with large, college level introductory psychology classes. Researcher bias may also 
exist, as this study was conceptualized and conducted by researchers who were also involved in 
delivering instruction to the experimental and control groups. Another limitation in this study 
may be a consequence of the fact that the sample was not randomly selected and the conditions 
were not randomly assigned, leading to unequal groups. Error in measurement may exist for 
research questions two through five, as the measure was researcher-generated and not field- 
tested for reliability and validity. 

Future research into the use of clickers should engage methods to compare procedures for 
embedding questions in lecture (e.g., pre-post lecture, throughout the lecture, etc.). To further 
explore the academic achievement of students using clickers, it would be worthwhile to link 
questions presented using the clicker system to questions on course tests, to detennine if 
utilization of the clickers contributes to specific knowledge acquisition. Examining the reliability 
and validity of the researcher created measure would decrease measurement error and lend 
credibility to the findings of students’ perceptions. 

V. Implications for Higher Education Administrators and Faculty. 

Since this research was conducted by professors engaging in scholarship of teaching and learning 
(Boyer, 1990), the researchers’ aim was to improve instructional practice within the college, 
while at the same time informing purchasing decisions of clickers. Currently, clickers are often 
used in large, college-level classes to increase student engagement, allowing teachers to poll 
classes by posing a question and providing an opportunity for students to respond in real-time via 
individual clickers. It appears that the utilization of clicker technology supports the generative 
theory of learning, by priming the cognitive processes used when learning, as demonstrated by 
improved academic perfonnance. Faculty of large, lecture-style classes seeking to engage 
students may be able to use clicker technology as one means of increasing generative learning, 
and therefore increasing student perfonnance. 

Yet, teachers, administrators, and students may question the efficacy of clickers because 
of the costs associated with adoption of the technology. Students are often required to purchase 
clickers, which range in cost from $10 to $40, while administrators must adopt the clickers, 
which require a substantial financial investment. Before assuming these costs and requiring 
students to make financial commitments, administrators are wise to ask whether or not the use of 
clickers increases student academic achievement, as well as what other options are available for 
formative assessment. Other options, such as Polleverywhere.com, provide instant audience 
feedback using mobile devices of audience members at significantly less expense (e.g., the cost 
of a text message). Polleverywhere.com provides free templates for PowerPoint slides which can 
be customized with questions about course content and embedded into existing PowerPoint 
presentations. Audience members are able to respond using SMS texting for the cost of a text 
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message, with no cost to the higher education institution for groups of up to 30 participants at a 
time, and $65 per month for 250 participants at a time. As new technologies develop, higher 
education administrators, instructors, and students have increasingly diverse options to improve 
academic achievement. New developments in technology should be evaluated to determine if 
perceived benefits outweigh the costs. 
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